fix keycloak missing resource error check by stephen-bracken · Pull Request #69028 · apache/airflow

stephen-bracken · 2026-06-26T10:52:08Z

Fix Auth manager check for keycloak 'resource not found' error

The error raised by keycloak that should be handled is {"error":"invalid_resource", "error_description": "Resource with id [Dag:team-a] does not exist."} This returns a 400 client error which is then converted into a 500 error by Airflow.

Sample log:

{"timestamp":"2026-06-26T10:37:18.612975Z","level":"error","event":"Exception in ASGI application\n","logger":"uvicorn.error","filename":"httptools_impl.py","lineno":425,"exception":[{"exc_type":"AirflowException","exc_value":"Request not recognized by Keycloak. invalid_resource. Resource with id [Dag:<team name>] does not exist.","exc_notes":[],"syntax_error":null,"is_cause":false,"frames":[{"filename":"/home/airflow/.local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py","lineno":420,"name":"run_asgi"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py","lineno":60,"name":"__call__"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/fastapi/applications.py","lineno":1159,"name":"__call__"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/applications.py","lineno":107,"name":"__call__"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/middleware/errors.py","lineno":186,"name":"__call__"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/middleware/errors.py","lineno":164,"name":"__call__"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/airflow/api_fastapi/common/http_access_log.py","lineno":83,"name":"__call__"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/middleware/gzip.py","lineno":29,"name":"__call__"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/middleware/gzip.py","lineno":130,"name":"__call__"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/middleware/gzip.py","lineno":46,"name":"__call__"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/middleware/base.py","lineno":191,"name":"__call__"},{"filename":"/usr/python/lib/python3.11/contextlib.py","lineno":158,"name":"__exit__"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/_utils.py","lineno":87,"name":"collapse_excgroups"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/middleware/base.py","lineno":193,"name":"__call__"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/airflow/api_fastapi/auth/middlewares/refresh_token.py","lineno":61,"name":"dispatch"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/middleware/base.py","lineno":168,"name":"call_next"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/middleware/base.py","lineno":144,"name":"coro"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/middleware/exceptions.py","lineno":63,"name":"__call__"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/_exception_handler.py","lineno":53,"name":"wrapped_app"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/_exception_handler.py","lineno":42,"name":"wrapped_app"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py","lineno":18,"name":"__call__"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/routing.py","lineno":716,"name":"__call__"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/routing.py","lineno":736,"name":"app"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/routing.py","lineno":290,"name":"handle"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/fastapi/routing.py","lineno":134,"name":"app"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/_exception_handler.py","lineno":53,"name":"wrapped_app"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/_exception_handler.py","lineno":42,"name":"wrapped_app"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/fastapi/routing.py","lineno":120,"name":"app"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/fastapi/routing.py","lineno":457,"name":"app"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/fastapi/dependencies/utils.py","lineno":680,"name":"solve_dependencies"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/concurrency.py","lineno":32,"name":"run_in_threadpool"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/anyio/to_thread.py","lineno":63,"name":"run_sync"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py","lineno":2518,"name":"run_sync_in_worker_thread"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py","lineno":1002,"name":"run"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/airflow/api_fastapi/core_api/security.py","lineno":262,"name":"depends_permitted_dags_filter"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/airflow/utils/session.py","lineno":100,"name":"wrapper"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/airflow/api_fastapi/auth/managers/base_auth_manager.py","lineno":578,"name":"get_authorized_dag_ids"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/keycloak/auth_manager/keycloak_auth_manager.py","lineno":455,"name":"filter_authorized_dag_ids"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/keycloak/auth_manager/cache.py","lineno":77,"name":"single_flight"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/keycloak/auth_manager/keycloak_auth_manager.py","lineno":453,"name":"query_keycloak"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/airflow/api_fastapi/auth/managers/base_auth_manager.py","lineno":612,"name":"filter_authorized_dag_ids"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/airflow/api_fastapi/auth/managers/base_auth_manager.py","lineno":612,"name":"<setcomp>"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/airflow/api_fastapi/auth/managers/base_auth_manager.py","lineno":608,"name":"_is_authorized_dag_id"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/keycloak/auth_manager/keycloak_auth_manager.py","lineno":229,"name":"is_authorized_dag"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/keycloak/auth_manager/keycloak_auth_manager.py","lineno":434,"name":"_is_authorized"}],"is_group":false,"exceptions":[]}]}

related: #68943
related: #68951

Was generative AI tooling used to co-author this PR?

Yes (please specify the tool below)
No

vincbeck · 2026-06-26T14:17:57Z

Tagging @onlyarnav here, you are modifying the constant KEYCLOAK_RESOURCE_NOT_FOUND_ERROR that @onlyarnav uses to match exception thrown by Keycloak when a resource does not exist. It might break things. @onlyarnav, can you review this one please?

stephen-bracken · 2026-06-26T15:21:05Z

@vincbeck Updating the constant should be fine as its only used with this function

stephen-bracken · 2026-06-26T16:32:54Z

@vincbeck @onlyarnav I've updated the PR to check against the error field from the keycloak json response for invalid_resource in the same way that the 400 response is handled here:

airflow/providers/keycloak/src/airflow/providers/keycloak/auth_manager/keycloak_auth_manager.py

Lines 440 to 443 in 4e2f5d4

    
           error = json.loads(resp.text) 
        
           raise AirflowException( 
        
               f"Request not recognized by Keycloak. {error.get('error')}. {error.get('error_description')}" 
        
           )

vincbeck · 2026-06-26T17:04:08Z

You really seem to override what @onlyarnav did, I'd like him to review it

onlyarnav · 2026-06-26T17:48:20Z

Thanks for the update! I just had a look at it.

My PR handled the error based on the original issue, where the response appeared to be a plain text string (resource not found: ...) rather than a JSON response. Since @stephen-bracken reported the original issue, I only had the information available there and wasn't aware that Keycloak could also return the structured invalid_resource JSON response.
Now that you shared the sample log, it's much clear.

From what you've shared, checking the structured JSON response via error == "invalid_resource" looks like a cleaner approach, and it aligns with the log you've posted.

My only question is whether the original plain-text resource not found: ... response is still possible on any supported Keycloak version or configuration. If that format can still occur, it might be worth handling both cases to avoid a regression. Otherwise, I'm happy with the updated approach.

stephen-bracken · 2026-06-27T11:46:34Z

hey @onlyarnav sorry for the confusion, I made a mistake in my original github issue. The error is as listed in this PR. Additionally the response from keycloak is a 400 which gets converted into a 500 by the Airflow webserver. I've updated the PR to reflect this.

see https://stackoverflow.com/questions/68417386/invalid-resource-or-resourceid-does-not-exist-keycloak

onlyarnav

Seems good to me. Thanks for the clarification regarding the error code.

The error raised by keycloak that should be handled is `{"error":"invalid_resource", "error_description": "Resource with id [Dag:team-a] does not exist."}`. This returns a 400 client error which is then converted into a 500 error by Airflow.

fix keycloak missing resource error check

b78a446

boring-cyborg Bot added area:providers provider:keycloak labels Jun 26, 2026

stephen-bracken mentioned this pull request Jun 26, 2026

Multi team: keycloak returns 400 error on /dags screen when teams are not defined in the keycloak client #68943

Closed

2 tasks

Stephen Bracken added 2 commits June 26, 2026 12:02

add doc note on multi team keycloak resources

91801c7

fix tests

ebc208e

stephen-bracken force-pushed the fix-keycloak-error branch from 72a2abf to ebc208e Compare June 26, 2026 11:34

fix linting

24b997b

stephen-bracken force-pushed the fix-keycloak-error branch from 84b07f0 to 24b997b Compare June 26, 2026 13:20

stephen-bracken marked this pull request as ready for review June 26, 2026 14:06

stephen-bracken requested review from bugraoz93 and vincbeck as code owners June 26, 2026 14:06

vincbeck approved these changes Jun 26, 2026

View reviewed changes

This comment was marked as outdated.

Sign in to view

onlyarnav mentioned this pull request Jun 26, 2026

Status of testing Providers that were prepared on June 26, 2026 (ad-hoc) #69049

Open

15 tasks

change error check to use keycloak 'error' response field

474b647

stephen-bracken force-pushed the fix-keycloak-error branch from 7af4a2a to 474b647 Compare June 26, 2026 22:09

stephen-bracken requested a review from onlyarnav June 26, 2026 22:09

stephen-bracken force-pushed the fix-keycloak-error branch 2 times, most recently from e64d9c1 to 393bb00 Compare June 27, 2026 11:55

catch 400 error from keycloak

46b767d

stephen-bracken force-pushed the fix-keycloak-error branch from 393bb00 to 46b767d Compare June 27, 2026 11:56

onlyarnav reviewed Jun 27, 2026

View reviewed changes

onlyarnav approved these changes Jun 27, 2026

View reviewed changes

vincbeck approved these changes Jun 29, 2026

View reviewed changes

vincbeck merged commit fa514df into apache:main Jun 29, 2026
77 checks passed

stephen-bracken deleted the fix-keycloak-error branch June 29, 2026 13:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix keycloak missing resource error check#69028

fix keycloak missing resource error check#69028
vincbeck merged 6 commits into
apache:mainfrom
stephen-bracken:fix-keycloak-error

stephen-bracken commented Jun 26, 2026 •

edited

Loading

Uh oh!

vincbeck commented Jun 26, 2026

Uh oh!

stephen-bracken commented Jun 26, 2026

Uh oh!

stephen-bracken commented Jun 26, 2026

Uh oh!

vincbeck commented Jun 26, 2026

Uh oh!

onlyarnav commented Jun 26, 2026

Uh oh!

This comment was marked as outdated.

Uh oh!

stephen-bracken commented Jun 27, 2026

Uh oh!

onlyarnav left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

stephen-bracken commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Fix Auth manager check for keycloak 'resource not found' error

Was generative AI tooling used to co-author this PR?

Uh oh!

vincbeck commented Jun 26, 2026

Uh oh!

stephen-bracken commented Jun 26, 2026

Uh oh!

stephen-bracken commented Jun 26, 2026

Uh oh!

vincbeck commented Jun 26, 2026

Uh oh!

onlyarnav commented Jun 26, 2026

Uh oh!

This comment was marked as outdated.

Uh oh!

stephen-bracken commented Jun 27, 2026

Uh oh!

onlyarnav left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

stephen-bracken commented Jun 26, 2026 •

edited

Loading